Skip to content

[v7r2] Run most unit tests for Python 3#4726

Closed
chrisburr wants to merge 43 commits into
DIRACGrid:integrationfrom
chrisburr:run-py3-pytest
Closed

[v7r2] Run most unit tests for Python 3#4726
chrisburr wants to merge 43 commits into
DIRACGrid:integrationfrom
chrisburr:run-py3-pytest

Conversation

@chrisburr

Copy link
Copy Markdown
Member

This builds upon #4718 to add support for actually running the unit tests instead of just collecting them. As #4718 hasn't been merged yet, see here for the diff chrisburr/DIRAC@py3-pytest-discover...chrisburr:run-py3-pytest. As usual for these changes, reviewing individual commits might be easier that the entire PR.

There is a lot left to port but I limited myself to the unit tests so we can be reasonably sure I haven't broken anything. I suspect after this is merged it shouldn't take much more to get a functional client installation running with Python 3.

This PR includes #4391 as I didn't want waste time looking for a Python 3 build that uses OpenSSL 1.0.2, @chaen can #4391 be unmarked as WIP now M2Crypto has been updated?

There are three tests which are still disabled as they fail:

  • test_BaseType_Unicode and test_nestedStructure fail due to DEncode's string and unicode types being poorly defined. I think this is hard to fix without major changes to DIRAC or backwads incompatible changes to DEncode itself. Hopefully we can ignore it until we've moved to JSON.
  • testLockedClass deadlocks frequently due to the use of self.__processThread._Thread__stop(). Officially "threads cannot be destroyed, stopped, suspended, resumed, or interrupted" so the code should be rewritten to follow the supported alternative: "If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event.".

I also add flaky to the CI as test_cpuUsage struggles to produce enough system CPU usage to be registered thanks performance improvements in Python 3. I think it's worthwhile so we can use it for the handful of other tests which are unreliable.

BEGINRELEASENOTES

*Python 3
NEW: Run most unit tests with Python 3

ENDRELEASENOTES

chrisburr and others added 30 commits August 16, 2020 23:09
Co-authored-by: fstagni <federico.stagni@cern.ch>
Comment thread Core/Utilities/test/Test_DAG.py Outdated
Comment thread Core/DISET/private/Transports/BaseTransport.py
Comment thread Core/Utilities/DEncode.py
Comment thread Core/Utilities/List.py
Comment thread WorkloadManagementSystem/Utilities/PilotWrapper.py Outdated
Comment thread Core/Utilities/test/Test_ProcessPool.py
Comment thread environment.yml
Comment thread ConfigurationSystem/Client/Utilities.py Outdated
Comment thread Core/Utilities/ClassAd/ClassAdLight.py
Comment thread Core/Utilities/test/Test_DAG.py Outdated
Comment thread Core/Utilities/test/Test_DAG.py Outdated

@andresailer andresailer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clicked through all the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants